Content-based and perceptual bit-allocation using matching pursuits

When communicating at a very-low bit-rate, video coders are unable to preserve high visual quality for all images. A selection of key regions according to human viewing and understanding may therefore be useful: it allows extracting essential features and to code them with a high quality, while the remainder of the image is coarsely transmitted. Such an approach requires a signal decomposition allowing an adaptive spatial variant bit allocation. Matching Pursuits (MP) is an iterative procedure leading to such a signal representation. Incorporated within a motion compensated video codec, MP allows for adaptive representation of the Displaced Frame Difference. Bit allocation procedures for Matching Pursuits differ from the ones used for DCTor other transformbased signal representations. For transformbased methods, the signal is first projected on a complete set of orthogonal base functions. Coefficients are then quantized. The quantization selects which information is transmitted. So, the selection is constrained by the chosen transform. On the contrary, MP explicitly select the information to be transmitted among a large and overcomplete set of functions and quantize it according to a fixed and constant step. For MP, bit allocation is equivalent to function selection. This makes Matching Pursuits well-suited for spatial variant bit allocation, since functions matched to important objects can be chosen. In this paper, we first show the abilities of Matching Pursuits to allocate an available bit-budget according to a semantic understanding of the scene. This semantic information can be provided either automatically or interactively. Second, the possibility to incorporate perceptive criteria within the MP coding algorithm is investigated.